fix(parser): preserve strikethrough and other non-enumerated styles - #18
Merged
Conversation
_extract_style() decided whether a cell was styled from a hand-written attribute list that omitted font.strikethrough and font.underline, so a cell whose only formatting was one of those had its entire CellStyle discarded — taking the correctly-parsed font with it. Compare against a default FontStyle instead of enumerating attributes. Each _extract_* helper already returns None when nothing is set, so the enumeration was redundant as well as lossy; dropping it also fixes fills with only bg_color and alignments with only text_rotation/indent. Also expose font_strikethrough on chunk cells in to_json(). Strikethrough commonly marks deprecated or void rows, so downstream RAG consumers need it alongside font_color/fill_color. Fixes #17 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Excel forbids overlapping merged regions, but malformed and
machine-generated files contain them, and _build_merge_lookup mapped
every cell of every region into one flat dict. A cell in an intersection
was written twice and the last region silently won, leaving the parse
output self-contradictory in two ways:
* the loser's master was a slave whose master was never flagged, so
every other cell in its region pointed at an unflagged master (M2)
* a cell that was master of one region and slave of another came back
flagged both (M4) and slave-with-no-master (M1), because CellParser
pre-flags any openpyxl MergedCell as a slave
Resolve overlaps up front: accept regions first-come in reading order
and drop any that intersect an accepted one, reporting each drop as a
WARNING instead of losing it silently. Reading order makes the survivor
independent of the order the file happens to list ranges in, so the
workbook hash stays deterministic, and surviving regions keep their
original ordering so well-formed workbooks are bit-identical.
Make the merge lookup the single authority for the flags while here: it
now sets master/slave exclusively and clears CellParser's provisional
slave flag for merges we do not track, which also covers stale ranges
that openpyxl reports outside any declared region.
Found by pointing the corpus robustness suite at real workbooks: 6 of
112 violated the merge invariants, all of them from this cause.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every corpus source was dead, and the download tests asserted only
isinstance(files, list) while download_and_extract_xlsx swallowed
RequestException into a log warning. So they reported green for months
having fetched nothing:
* 3x SheetJS test_files -> the submodule repo is blocked by GitHub
* openpyxl genuine/empty.xlsx -> 404 (openpyxl is not on GitHub)
* Zenodo EUSES.zip -> 404, record withdrawn
* Enron xls dump -> 200, but 321 MB for ~0 .xlsx, as its own docstring
predicted
tests/fixtures/corpus/ therefore stayed empty, and because corpus_files
is computed at import time the eight robustness tests that depend on it
skipped silently. The suite spent 5.5 minutes verifying nothing.
Repoint at sources chosen for durability over volume: sdists of
Excel-handling PyPI packages (published artifacts are immutable, so
these cannot rot) and upstream parser fixture directories enumerated
through the GitHub contents API, so a rename upstream costs one file
rather than the whole corpus. 80 files in 7.5s, down from 0 in 325s.
Assert on what arrived. A genuinely offline machine skips; a reachable
source returning nothing usable now fails. Members that are not real
XLSX containers are skipped at extraction: upstream suites ship a
zero-byte file and an encrypted OLE workbook under an .xlsx name, which
belong in targeted unit tests, not in a corpus asserting that
well-formed workbooks parse.
Split the two contracts the corpus was conflating. Upstream fixture
suites include files that are valid ZIPs but not valid OOXML packages
(missing [Content_Types].xml, missing sharedStrings.xml, a style
attribute openpyxl rejects). openpyxl cannot open them either, so
test_has_sheets was asserting the impossible; it now skips them, and
test_unloadable_files_report_an_error covers the contract that does
apply — degrade with a recorded ERROR rather than crash or come back
silently empty. test_success_rate's 95% threshold likewise measures the
loadable subset, so it tracks this parser's health rather than how many
negative fixtures upstream happens to ship.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two defects compounded here, and each hid the other. _has_meaningful_style() enumerated three font attributes, so it missed strikethrough, underline, font size and font name — the same blind spot just fixed in _extract_style, in the one place where a false negative discards data outright. It was also wrong in the opposite direction: building a partial Font leaves `color` as None while the inherited default font has one, so an explicitly formatted cell looked *less* styled than an untouched neighbour, and every untouched cell reported as styled. Ask openpyxl instead of re-deriving it. A cell's format is recorded as non-zero style ids for exactly the aspects that differ from the workbook default, which is both precise and immune to growing another blind spot as style support widens. Verified against one cell per aspect plus an untouched control: correct on all ten, where comparing resolved style objects to fresh defaults is not (a loaded default font does not equal Font()). The predicate was moot regardless: it spared a valueless cell from the early `continue`, and then the gate that stores cells dropped anything is_empty. So styling on empty cells was parsed and thrown away. That gate now keeps a cell that carries a style. Strikethrough or a fill on an otherwise blank row is precisely the "deprecated / void" marker that motivated #17, and dropping it silently is the same data loss one layer down. Correcting the predicate also removed an accident the old one was holding up. A merge master arrives as an ordinary Cell, not a MergedCell, so the empty-cell skip never exempted it; an empty master survived only because the broken predicate called every untouched cell styled. Reporting those cells honestly started dropping empty masters, stranding every slave in the region with a merge_master that no longer existed — 348 M2 violations across 44 corpus workbooks, none of which the default suite could see. The skip now spares any cell taking part in a merge, via the merge lookup for the master. Costs 3.6% more cells across the 192-workbook corpus (9,013 on 251,376). Cells kept this way still report is_empty, so nothing mistakes them for data. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
arnav2
marked this pull request as ready for review
September 2, 2026 21:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #17.
The reported bug was real and is fixed. Verifying it properly meant running the corpus suite that was supposed to cover this class of regression — which turned out to be inert, and which then surfaced two more parser bugs of the same family. Four commits, each independently reviewable.
1.
fix(parser): strikethrough-only cells keep their style_extract_style()decided whether a cell was styled from a hand-written attribute list that omittedfont.strikethroughandfont.underline, so a cell whose only formatting was one of those had its entireCellStylediscarded — taking the correctly-parsed font with it.Rather than adding the two missing attributes as the issue suggested,
has_stylenow compares the parsed font against a defaultFontStyle()sentinel. Each_extract_*helper already returnsNonewhen nothing is set, so the enumeration was redundant as well as lossy. That closes the whole class of bug: it also recovers fills with onlybg_colorand alignments with onlytext_rotation/indent, which the old list dropped too.The secondary request is included —
font_strikethroughis now exposed on chunk cells into_json(), alongsidefont_color/fill_color.Reproducer from the issue now prints:
2.
fix(parser): merge invariants hold when regions overlapFound by this work, not reported. Excel forbids overlapping merged regions, but malformed and machine-generated files contain them, and
_build_merge_lookupmapped every cell of every region into one flat dict. A cell in an intersection was written twice and the last region silently won, leaving the parse output self-contradictory two ways:M2)M4) and slave-with-no-master (M1), becauseCellParserpre-flags any openpyxlMergedCellas a slaveOverlaps are now resolved up front: regions are accepted first-come in reading order, any that intersect an accepted one is dropped, and each drop is reported as a
WARNINGinstead of being lost silently. Reading order makes the survivor independent of the order the file lists ranges in, so the workbook hash stays deterministic; surviving regions keep their original ordering, so well-formed workbooks are bit-identical.The merge lookup is now also the single authority for the flags — it sets master/slave exclusively and clears
CellParser's provisional slave flag for merges we don't track, which additionally covers stale ranges openpyxl reports outside any declared region.6 of 112 real workbooks were affected. Confirmed pre-existing by running
check_invariantsagainst the parser at4834382: identical violation counts (2, 4, 13, 2, 2, 16) before and after, all now zero.3.
fix(parser): formatting on valueless cells is preservedFound by this work, not reported. The same enumeration blind spot as commit 1, in the one place where a false negative discards data outright — and compounded by a second defect that hid it.
_has_meaningful_style()checked three font attributes, so it missed strikethrough, underline, font size and font name. It was also wrong in the opposite direction: building a partialFontleavescolorasNonewhile the inherited default font has one, so an explicitly formatted cell looked less styled than an untouched neighbour, and every untouched cell reported as styled.It now asks openpyxl rather than re-deriving. A cell's format is recorded as non-zero style ids for exactly the aspects that differ from the workbook default — precise, and immune to growing another blind spot as style support widens. Validated against one cell per aspect plus an untouched control: correct on all ten, where comparing resolved style objects against fresh defaults is not (a loaded default font does not equal
Font()).The predicate was moot either way: it spared a valueless cell from the early
continue, then the gate that stores cells dropped anythingis_empty. Styling on empty cells was parsed and then thrown away. That gate now keeps a cell carrying a style — strikethrough or a fill on an otherwise blank row is precisely the "deprecated / void" marker that motivated this issue, and dropping it silently is the same data loss one layer down.Correcting the predicate also removed an accident the old one was holding up, and this is the part worth reviewing closely. A merge master arrives as an ordinary
Cell, not aMergedCell, so the empty-cell skip never exempted it — an empty master survived only because the broken predicate called every untouched cell styled. Reporting those cells honestly started dropping empty masters, stranding every slave in the region with amerge_masterthat no longer existed: 348M2violations across 44 corpus workbooks, none of which the default suite could see. The skip now spares any cell taking part in a merge, recognising the master via the merge lookup.Costs 3.6% more cells across the 192-workbook corpus (9,013 on 251,376). Cells kept this way still report
is_empty, so nothing mistakes them for data.4.
test(corpus): make the corpus suite actually exercise the parserEvery corpus source was dead, and the download tests asserted only
isinstance(files, list)whiledownload_and_extract_xlsxswallowedRequestExceptioninto a log warning. They reported green having fetched nothing:test_filesgenuine/empty.xlsxEUSES.zip.xlsdump.xlsx, as its own docstring predictedSo
tests/fixtures/corpus/stayed empty, and becausecorpus_filesis computed at import time, the eight robustness tests depending on it skipped silently. The suite spent 5.5 minutes verifying nothing.Repointed at sources chosen for durability over volume: sdists of Excel-handling PyPI packages (published artifacts are immutable, so they can't rot) and upstream parser fixture directories enumerated through the GitHub contents API, so an upstream rename costs one file rather than the whole corpus. 80 files in 7.5s, down from 0 in 325s.
The tests now assert on what arrived. A genuinely offline machine skips; a reachable source returning nothing usable fails. Non-container members are skipped at extraction — upstream suites ship a zero-byte file and an encrypted OLE workbook under an
.xlsxname.Two contracts the corpus was conflating are now split. Upstream fixture suites include files that are valid ZIPs but not valid OOXML packages (missing
[Content_Types].xml, missingsharedStrings.xml, a style attribute openpyxl rejects). openpyxl cannot open them either, sotest_has_sheetswas asserting the impossible; it skips them now, andtest_unloadable_files_report_an_errorcovers the contract that does apply — degrade with a recordedERRORrather than crash or come back silently empty.test_success_rate's 95% threshold likewise measures the loadable subset, so it tracks this parser's health rather than how many negative fixtures upstream happens to ship.Verification
sheet_parser.pyare pre-existing (missing stubs, three unrelated spots)Note
The
ruffandmypyCI jobs are still non-gating (|| true), and the mypy job targetssrc/ks_xlsx_parser, which no longer exists after theexcel_parserrename — so it currently checks nothing. Untouched here.🤖 Generated with Claude Code